Skip to content

CockroachDB Support#388

Merged
devhawk merged 28 commits into
mainfrom
devhawk/crdb
May 18, 2026
Merged

CockroachDB Support#388
devhawk merged 28 commits into
mainfrom
devhawk/crdb

Conversation

@devhawk
Copy link
Copy Markdown
Collaborator

@devhawk devhawk commented May 18, 2026

This PR adds CockroachDB support. It also generally revamps how SysDB is organized and how Listen/Notify events are delivered.

Changes:

  • All DAO classes moved into database/dao directory
  • Added DbContext to carry data source, schema, serializer and a closed check in a single DAO parameter
  • Added new SysDB signal infrastructure (SignalMap, SignalKey, Subscription)
  • Rewrote recv and getEvent to use new signal infrastructure
  • Added DBOSConfig useListenNotify field
  • Added cockroach DB check in sysdb ctor to override useListenNotify value
  • renamed NotificationService -> NotificationListenerSource and reworked to use new signal infrastructure
  • added NotifcationRegistry and NotificationSource interfaces
  • added NullNotificationSource to use when useListenNotify is false or running on CockroachDB
  • Updated migration manager to split migration 1 into two parts (listen/notify trigger + function and every thing else) and move migration 10 check to java for CRDB compat
  • reworked test infrastructure to truncate DBOS tables rather than dropping the DB to save on migration execution time (very very very slow on CRDB)
  • added DBOS_TEST_USE_COCKROACH_DB env var to run test suite on CRDB instead of PG
  • Added ParallelExecutionConfigurationStrategy implementation to use fixed config for CRDB since those containers are much more resource intensive than the PG containers

@devhawk devhawk marked this pull request as ready for review May 18, 2026 06:28
@devhawk devhawk requested review from Copilot, kraftp and maxdml May 18, 2026 14:46
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds CockroachDB compatibility to the system database layer and refactors how SysDB data access and event delivery are implemented (including a new signal/subscription abstraction and a configurable LISTEN/NOTIFY vs polling mode).

Changes:

  • Introduces useListenNotify configuration and CockroachDB detection to automatically disable LISTEN/NOTIFY where unsupported.
  • Refactors SysDB access into DAO classes using a shared DbContext, and rewrites recv / getEvent around a new signal infrastructure (SignalMap, SignalKey, Subscription).
  • Updates test infrastructure and CI to run against CockroachDB (container support, migrations expectations, and parallelism tuning).

Reviewed changes

Copilot reviewed 46 out of 46 changed files in this pull request and generated 11 comments.

Show a summary per file
File Description
transact/src/test/resources/junit-platform.properties Switches JUnit parallel execution strategy to a custom strategy (for CRDB tuning).
transact/src/test/java/dev/dbos/transact/utils/PgContainer.java Adds CRDB container support and changes test DB lifecycle to reuse + truncate instead of drop/recreate.
transact/src/test/java/dev/dbos/transact/utils/CrdbParallelExecutionConfigurationStrategy.java Limits parallelism when running tests against CRDB.
transact/src/test/java/dev/dbos/transact/txstep/JdbcStepFactoryTest.java Ensures required DB objects are present/clean when reusing pooled DBs.
transact/src/test/java/dev/dbos/transact/txstep/JdbcStepFactoryInitTest.java Uses a fresh container and ensures database exists before running tests.
transact/src/test/java/dev/dbos/transact/migrations/MigrationManagerTest.java Updates migration assertions for CRDB vs PG and adds coverage for notify-disabled migrations.
transact/src/test/java/dev/dbos/transact/migrations/CockroachMigrationTest.java Adds explicit PG vs CRDB migration behavior tests (LISTEN/NOTIFY objects).
transact/src/test/java/dev/dbos/transact/invocation/CustomSchemaTest.java Uses a fresh container for schema tests.
transact/src/test/java/dev/dbos/transact/execution/DBOSExecutorTest.java Switches JDK gating from env vars to runtime feature checks.
transact/src/test/java/dev/dbos/transact/database/SignalMapTest.java Adds tests for the new signal/subscription mechanism.
transact/src/test/java/dev/dbos/transact/database/ImportExportTest.java Updates to new step result recording API.
transact/src/test/java/dev/dbos/transact/config/ConfigTest.java Ensures DB existence for config tests that provide a prebuilt DataSource.
transact/src/test/java/dev/dbos/transact/client/PgSqlClientTest.java Updates function invocation parameter ordering/signatures.
transact/src/main/java/dev/dbos/transact/workflow/internal/StepResult.java Adds helpers to build step results from serialized output/error.
transact/src/main/java/dev/dbos/transact/migrations/MigrationManager.java Adds CRDB-aware LISTEN/NOTIFY migration selection and moves migration-10 PK check into Java.
transact/src/main/java/dev/dbos/transact/execution/DBOSExecutor.java Updates event/step APIs to new caller context and step result methods.
transact/src/main/java/dev/dbos/transact/DBOSClient.java Adds a constructor parameter to control LISTEN/NOTIFY usage.
transact/src/main/java/dev/dbos/transact/database/SystemDatabase.java Introduces DbContext, closed-checking, CRDB detection, and pluggable notification source.
transact/src/main/java/dev/dbos/transact/database/signal/Subscription.java Adds an AutoCloseable subscription future type.
transact/src/main/java/dev/dbos/transact/database/signal/SignalMap.java Adds a concurrent signal/subscribe map for wakeups.
transact/src/main/java/dev/dbos/transact/database/signal/SignalKey.java Adds typed keys and wake reasons for signals.
transact/src/main/java/dev/dbos/transact/database/NotificationListenerSource.java Replaces the prior notification service with a signal-map-based listener.
transact/src/main/java/dev/dbos/transact/database/GetWorkflowEventContext.java Removes the old getEvent caller context record.
transact/src/main/java/dev/dbos/transact/database/GetEventCaller.java Adds the new getEvent caller context record.
transact/src/main/java/dev/dbos/transact/database/DbContext.java Adds a shared context object for DAOs (datasource/schema/serializer/closed check).
transact/src/main/java/dev/dbos/transact/database/dao/WorkflowDAO.java Moves DAO into database/dao and refactors to accept DbContext.
transact/src/main/java/dev/dbos/transact/database/dao/StreamsDAO.java Moves DAO into database/dao and refactors to accept DbContext.
transact/src/main/java/dev/dbos/transact/database/dao/StepsDAO.java Moves DAO into database/dao and refactors to accept DbContext.
transact/src/main/java/dev/dbos/transact/database/dao/SchedulesDAO.java Moves DAO into database/dao and refactors to accept DbContext.
transact/src/main/java/dev/dbos/transact/database/dao/QueuesDAO.java Moves DAO into database/dao and refactors to accept DbContext.
transact/src/main/java/dev/dbos/transact/database/dao/NotificationsDAO.java Refactors recv/getEvent to new signal infra and DbContext + notification source abstractions.
transact/src/main/java/dev/dbos/transact/database/dao/ExternalStateDAO.java Moves DAO into database/dao and refactors to accept DbContext.
transact/src/main/java/dev/dbos/transact/database/dao/ApplicationVersionDAO.java Moves DAO into database/dao and refactors to accept DbContext.
transact/src/main/java/dev/dbos/transact/config/DBOSConfig.java Adds useListenNotify to configuration and Spring/CLI integration points.
transact/build.gradle.kts Adds test dependencies for JUnit platform engine and CockroachDB testcontainers.
transact-spring-boot-starter/src/test/java/dev/dbos/transact/spring/PgContainer.java Updates testcontainer image tag.
transact-spring-boot-starter/src/main/java/dev/dbos/transact/spring/DBOSProperties.java Adds Spring property useListenNotify.
transact-spring-boot-starter/src/main/java/dev/dbos/transact/spring/DBOSAutoConfiguration.java Wires Spring property into DBOSConfig.
transact-jooq-step-factory/src/test/java/dev/dbos/transact/utils/PgContainer.java Updates testcontainer image tag.
transact-jdbi-step-factory/src/test/java/dev/dbos/transact/utils/PgContainer.java Updates testcontainer image tag.
transact-cli/src/test/java/dev/dbos/transact/cli/PgContainer.java Updates testcontainer image tag.
transact-cli/src/main/java/dev/dbos/transact/cli/MigrateCommand.java Updates migration invocation signature (adds useListenNotify parameter).
gradle/libs.versions.toml Adds JUnit platform engine + CockroachDB testcontainers aliases.
.github/workflows/test_crdb.yml Adds a dedicated CI workflow to run the suite against CockroachDB.
.github/workflows/on_push.yml Runs the CRDB workflow on push.
.github/workflows/on_pr.yml Runs the CRDB workflow on PRs.
Comments suppressed due to low confidence (2)

transact/src/main/java/dev/dbos/transact/database/dao/NotificationsDAO.java:490

  • When getEvent times out, result.value() is null and the step result is recorded with output=null. On replay, StepResult.toResult() throws because both output and error are null, so a timed-out getEvent inside a workflow will not be replayable.

Record an explicit serialized null (e.g., SerializationUtil.serializeValue(null, ..., ctx.serializer())) so output is non-null, or adjust the replay logic to treat (output==null && error==null) as a recorded null result for this step type.
transact/src/test/java/dev/dbos/transact/utils/PgContainer.java:34

  • Using floating :latest image tags makes the test environment non-reproducible and can introduce sudden breakages when upstream images change.

Pin these to known-good versions (for both PostgreSQL and CockroachDB) so CI and local runs are stable.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread transact/src/main/java/dev/dbos/transact/database/SystemDatabase.java Outdated
Comment thread transact/src/main/java/dev/dbos/transact/database/SystemDatabase.java Outdated
Comment thread transact/src/test/java/dev/dbos/transact/utils/PgContainer.java
Comment thread .github/workflows/test_crdb.yml
Comment thread transact-cli/src/test/java/dev/dbos/transact/cli/PgContainer.java
Comment thread .github/workflows/on_pr.yml
Comment thread transact/src/test/java/dev/dbos/transact/utils/PgContainer.java
@devhawk devhawk merged commit f4f5607 into main May 18, 2026
31 of 34 checks passed
@devhawk devhawk deleted the devhawk/crdb branch May 18, 2026 17:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants